HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism

Recently, some challenging tasks in multi-agent systems have been solved by hierarchical reinforcement learning methods. Inspired the intra-level and inter-level coordination human nervous system, we propose a novel value decomposition framework HAVEN based on for fully cooperative problems. To address instability arising from concurrent optimization of policies between various levels agents, introduce dual mechanism inter-agent strategies designing reward functions two-level hierarchy. does not require domain knowledge pre-training, can be applied to any variant. Our method achieves desirable results different decentralized partially observable Markov decision process domains outperforms other popular algorithms.